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Abstract. List decoding of Hcrmitian codes is reformulated to allow an ef- 
ficient and simple algorithm for the interpolation step. The algorithm is de- 
veloped using the theory of Grobner bases of modules. The computational 
complexity of the algorithm seems comparable to previously known algorithms 
achieving the same task, and the algorithm is better suited for hardware im- 
plementation. 



1. Introduction 

Following Sudan's idea of list decoding of Reed-Solomon codes ^5] > Shokrollahi 
and Wasserman presented the first form of list decoding of algebraic geometry- 
codes. Soon afterward, Guruswami and Sudan j5J added the notion of multiplicities 
to Shokrollahi and Wasserman's formulation, improving significantly the capability 
of list decoding. By these works, the current form of list decoding of algebraic 
geometry codes, consisting of an interpolation step and a root-finding step, was 
established. 

Subsequently, many efforts followed to develop practical algorithms for the in- 
terpolation step and the root-finding step. H0holdt and Nielsen [7] worked out 
explicitly an interpolation algorithm and a factorization algorithm specifically for 
Hermitian codes. Augot and Pecquet [Q, Gao and Shokrollahi 5 , and Wu and 
Siegel |17| presented efficient factorization or root-finding algorithms over func- 
tion fields. Sakata presented a fast interpolation method using the well-known 
Berlekamp-Massey-Sakata algorithm. Olshevsky and Shokrollahi JU| derived fast 
interpolation algorithms using the concept of displacement rank of structured ma- 
trices. 

Hermitian codes have been the most prominent example of algebraic geome- 
try codes, and a serious competitor of Reed-Solomon codes. First of all, they are 
significantly longer than Reed-Solomon codes for a fixed alphabet size, and they 
have good dimension and minimum distance properties. They also possess a rich 
algebraic and geometric structure that yields efficient encoding and decoding algo- 
rithms. New developments on decoding algorithms were often applied to Hermitian 
codes foremost, and an idea successful with Hermitian codes is likely to be extended 
for a general class of algebraic geometry codes. 

The aim of this paper is to extend the results of Lee and O'Sullivan [5j for 
Hcrmitian codes. This is a natural but, we think, nontrivial task. We needed to 



2000 Mathematics Subject Classification. 94B35,11T71. 

Key words and phrases. Hermitian codes; List decoding; Grobner bases; Interpolation 
algorithms. 

The first author was supported by the Korea Research Foundation Grant funded by the Korea 
Government (MOEHRD, Basic Research Promotion Fund) (KRF-2005-214-C00009). 

1 



2 



KWANKYU LEE AND MICHAEL E. O'SULLIVAN 



reformulate list decoding of Hcrmitian codes in the language of commutative alge- 
bra and Grobner bases. An advantage of the new formulation is to eliminate the 
computation of the "increasing zero bases" of a linear space as in [7j- The new 
formulation allows us to present a simple and efficient algorithm for the interpola- 
tion step using Grobner bases of modules. The algorithm is a natural adaptation 
to Hermitian codes of the algorithm for Reed-Solomon codes developed in [§]. 

In Section 2, we review basic properties of Hermitian curves and codes. Fulton 
0], Stichtcnoth (T£|, and Pretzel ^21 are our basic references for further informa- 
tion. In later sections, a basic understanding of Grobner bases is assumed. For 
an introduction to the theory, see Cox et al. [HE]. I n Section 3, we formulate list 
decoding of Hermitian codes. In Section 4, we decribe a method to find an optimal 
interpolation polynomial, namely the Q-polynomial. In Section 5, an efficient algo- 
rithm for the interpolation step is presented. In Section 6, some upper bounds for 
the Q-polynomial are given. In the appendix, we present an algorithm computing a 
Grobner basis for a module with a special set of generators, with respect to a special 
weight monomial order. It is a slight abstraction of Algorithm G for list decoding 
of Reed-Solomon codes presented in [Jj], and applicable for Hermitian codes as well. 

2. Codes on Hermitian curves 

Let F denote a finite field with q 2 elements. Let H C Ap be the Hermitian plane 
curve defined by the absolutely irreducible polynomial X q+1 — Y q — Y over F. The 
coordinate ring of H is the integral domain 

R = F[X, Y]/(X q+1 -Y q -Y). 

The function field of H is the quotient field K of R. Let x and y denote the residue 
classes of X and Y in R, respectively. So x q+1 — y q — y = 0, and R = ¥[x, y]. 

There are q 3 rational points on H , which are enumerated as P±, P%, . . . , P n with 
n = q 3 . The projective closure of H is a nonsingular curve with a unique rational 
point Poo at infinity. The functions x and y on H have poles at P^ of orders q and 
3 + 1, respectively. The genus of H is given by g = q{q — l)/2. 

The linear space C(uPao) for u > has a basis consisting of x % yi for Q < i, 
< j < q — 1, and qi + (q + l)j < u. Moreover 

oo 

R= |J £(«Poo) - F-arV. 

«=0 0<i 

0<J<q-l 

Recall that the Hamming space F™ is equpped with the Hamming distance d. 
Let Pi — (ai,(3i) for 1 < i < n. The evaluation map ev : R — > F™ defined by 

is a linear map over F. We now fix a positive integer u. Hermitian code C u is 
defined to be the linear code given as the image of C(uPoq) by the evaluation map. 
If u < n, then ev is injective on C(uPoo)i and the dimension of C u is equal to 
dimF(£(u-Poo))i which is U + 1 — g for u > 2g by the Riemann-Roch theorem. Note 
also that the minimum distance of C u is at least n — u. 

Let k denote the dimension of C u . For encoding, fix a basis of £(uPoo), say 
{(pi, <p2, . , . , (fk}- Then a message to = (wi, 0J2, ■ ■ ■ , Wfc) £ F fc is encoded to the 



LIST DECODING OF HERMITIAN CODES USING GROBNER BASES 



3 



codeword 

k 

c = ev(/u), m = y^^Ujifj. 

i=i 

We call fj, the message function corresponding to the codeword c. 

Example. Let q = 2. We consider the Hermitian curve H defined by X 3 + Y 2 + Y 
over F 4 = {0, a, a 2 , a 3 }. There are 8 rational points on H, 

(0, 0), (0, 1), (1, a), (1, a 2 ), (a, a), (a, a 2 ), (a 2 , a), (a 2 , a 2 ). 

Let u = 4. The linear space £(4P oc ) has a basis {1, x, y, x 2 }. Hermitian code C4 is 
an [8, 4, 4] linear code over F 4 . Our message is u — (a 2 , a 2 , 0, a 2 ), which is encoded 
to the codeword 

ev(fj,) = (a 2 , a 2 , a 2 , a 2 , 0,0, 0,0), 
where [i = a 2 + a 2 x + a 2 x 2 . We will continue this example throughout. 
Define 

(X^-X)(Y^ + Y-^-f3 t ) 

Hi ~ -A) G 1,1 

and let hi denote the residue class of Hi in R for 1 < i < n. 

For v = (vi, V2, ■ ■ ■ , v n ) G F™, define H v = Y^i=i v i^i^ an d ^ h v denote the 
residue class of H v in R. That is, 

n 

hy=y* j Villi. 
i=l 

Lemma 1. hi(Pj) = 1 if j = i, and otherwise. So ev(h v ) = v for v G F™. 
Proof. Recall that 

]J(X-a) =X* 2 -X, 11 (Y-b)=Y«+Y-F'-0 

a<EF b£¥ 

b"+b=f3"+l3 

for any (3 G F. For 1 < i < n, define 

(Xi 2 -X){Y« + Y -Pi 

1 ~U) = 



Hi=H(X-a) I] ( Y - b )= (X-a-)(Y-8-) 



It is immediate from the definition that Hi(aj,Pj) = for j ^ i. Taking partial 
derivatives with respect to X and Y of both sides of the equation 

(X - Oi)(Y - Pi) Hi = (Xi 2 -X)(Y* + Y- PI - pi) 

and substituting X and Y with a>i and Pi, we see that Hi(eti, pi) = — 1. As /ij is 
the residue class of — #j in R, the assertion follows. □ 
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Example (continued). The functions hi are as follows: 
hi = (x 3 + l)y + x 3 + 1, 
h 2 = (x 3 + l)y, 

h 3 = (x 3 + x 2 + x)y + a 2 x 3 + a 2 x 2 + a 2 x, 
hi = (x 3 + x 2 + x)y + ax 3 + ax 2 + ax, 
h 5 — (x 3 + ax 2 + a 2 x)y + a 2 x 3 + x 2 + ax. 
ha = (x 3 + ax 2 + a 2 x)y + ax 3 + a 2 x 2 + x, 

o 2 2 \ 23 2 

/17 = (x + a x + ax)y + a x + ax + x, 
hs = (x 3 + a 2 x 2 + ax)y + ax 3 + x 2 + a 2 x. 

Lastly, define 77 to be the residue class of riaeF^ ~ a ) = ^ q2 — X in R. So 

2 

rj = x q — x. 

3. List Decoding of Hermitian Codes 

We prove some lemmas required for a fundamental theorem, Theorem of list 
decoding of Hermitian codes. First, note that the surface S = H x has coordinate 
ring 

R[z] = FLY, Y, Z}/(X q+1 - Y q - Y), 
where z denotes the residue class of Z in the quotient ring. 
Lemma 2. Let m be a positive integer. Let v be a vector in F™. Then 

diniF R[z]/ (z — h v ,7]) m = n 

Let fj, £ R with t = d(v,ev(fj,)). Then 

dim r R[z]/((z - h v ,rj) m + (z - fi)) = m(n - t). 

Proof. Let k be an algebraic closure of F. We consider the ideal 

/ = (X q+1 -Y q -Y) + (Z - H v ,X q2 - X) m 

of k[X, Y, Z]. We claim that the zero set V(T) of / is {(ai,(3i,Vi) \ 1 < i < n}. As 
one inclusion is easily verified, we show that every (a, b, c) € V(I) equals (ai,(3i, Vi) 
for some 1 < i < n. Suppose (a, b, c) £ V(I). Since (X q — X) m G /, we have 
a q - a = 0. So a S F. We also have a q+1 - b q - b = 0. Taking the q-th power 
of the equation, we get a q+1 — b q ~ — b q = 0. Subtracting the second equation from 
the first, we get b q —6 = 0. Therefore b G F. Thus (a, b) must be one of the 
rational points on the Hermitian curve. Let (a, b) = Pi for some 1 < i < n. As 
(Z — H v ) m G /, we see c = u,-. The claim is now proved. 

As V(I) is finite, we have a natural isomorphism (see Theorem 2.2 in Chapter 4 
of 0) 

n 

k[X, Y, Z]/I = (auf3i<Vi) /IO (ait/3itVi) 

i=l 

where 0(a. it p itV A denotes the local ring fc[X, F, Z]/ X - ai ,Y—Pi,z-Vi)- ^ ow fi x The 
automorphism of k[X, Y, Z] given by (X, Y,Z) 1— » (A + ct^, y + ft, Z + Uj) induces 
the isomorphism 
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where O = k[X,Y, Z](x,y.z)> an d I' is the ideal 

/' = (X q+1 + (a? + a t )X -Y q -Y) + (Z + AX + BY, X q2 - X) m 
for some A,B £ k[X, Y\. As V(I') is finite and contains the origin, we have 

dimfe O/I'O = dim fe k[[X, Y, Z]]/I'k[[X, Y, Z]]. 
In k[[X, Y, Z]], we can write by the Weierstrass Preparation Theorem, 

Y q + Y - X q+1 - (a? + ai)X = (Y - XP)U 
for some P £ k[[X}] and a unit U of k[[X, Y]]. As an ideal of k[[X, Y, Z]}, 

l'k[[X, Y, Z\] = (Y — XP) + (Z + (A + BP)X, X q2 - X) m 
= (Y — XP) + (Z + (A + BP)X, X) m 
= (Y - XP) + (Z,X) m . 
So k[[X, Y, Z]]/I'k[[X, Y, Z\\ m k[X, Z]/(X, Z) m . Since this is true for all i, 

(m + 1 x 

The first assertion of the lemma now follows since 

dim F R[z]/(z - h v , r)) m = dim fe k[X, Y, Z]/I 

as I is an ideal generated by polynomials defined over F. 

The second assertion is proved similarly. So we will be brief, omitting repeated 
details. Let 

J = (X q+1 -Y q -Y) + (Z- H v , X q2 - X) m + {Z- M), 

where M £ V[X, Y] is such that /i is the residue class of M in R. Let ev(/i) = 
(ci,C2, • • • ,c„). Then V(J) = {(ai, 0i,Vi) \ Vi = Ci, 1 < i < n}. We have a natural 
isomorphism 

k{X,YZ]/J= o {ai ^ Vi) /JO iaiM . 

Vi — Ci 

Fix i with V{ = Ci. Then 0( ait p itVi )/ JO( ait p itVi ) is isomorphic to O/J'O, where 

J' = (X q+1 + (a q + ai )X -Y q -Y) + {Z + AX + BY, X ql - X) m 
+ (Z + CX + DY) 
for some A, B,C,D £ k[X, Y). Again 

dim fe O/J'O = dimfe k[[X, Y, Z]]/J'k[[X, Y, Z]], 

but now 

j'k[[X, Y, Z]] = (Y — XP) + (X) m + (Z + SX) 
for some P, S £ k[[X}]. This gives an isomorphism 

k[[X, Y, Z]]/j'k[[X, Y, Z\\ = k[X]/(X) m . 

Therefore 

dim k k[X,Y,Z}/ J = dim fe k \ X \l( X ) m = m(n-t), 

Vi=Ci 

from which the second assertion of the lemma follows. □ 
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Lemma 3. Let ip be a nonzero element in R. Then 

dimF(i?/V-0 = — u PooW- 

Proof. Recall that Poo is the unique point at infinity of the smooth curve H. There- 
fore 

R= f) P . 

Consider the homomorphism 

which maps p G R to ^5 in Op/ipOp for each P ^ P^. If tp G ker(g), then tp/tp G Op 
for P ^ Poo, which implies tp/ip G R, and hence <p G ?/>P. Therefore ker(g) = ^>P. 
To prove surjectivity, let S be the finite set of points of H at which vp(ip) > 0. 
Let (xp) be an element of the direct sum. Then by the Strong Approximation 
Theorem, there is a <p in the function field K such that v P (tp — \p) = v P (ip) for 
P G S and u P (» > for P £ 5 and P ^ P^. Then 

G P| Op = P, and tp = \P m °d V'Op 

for P 7^ Poo. This shows that g is surjective. 
Hence we have a natural isomophism 

0pM?p, 

which implies 

dim F (P/V0 = dim F (Op/i/>Op) = ^ v P (tp) = -v Pao {ip). 

□ 

We introduce two notations. For / G R[z], the ii-weighted degree of / is defined 
to be 

degj/) = max {-v Pca (ipi) + ui) 

0<i<a 

if / = ip a z a + - ■ ■+tpiz + tp . For / G R[z] and tp G P, we denote by /(<p) the element 
in P that is obtained by substituting z with tp in /. Observe that if <^ G £(uPoo), 
then -v Poo (/(?)) <deg u (/). 

Now we are ready to present a fundamental theorem upon which list decoding 
of Hermitian codes is based. Suppose that some codeword of C u was sent through 
a noisy channel. Let v denote the vector in ¥ n that was received by hard-decision 
on the channel output. Fix a positive integer to, called the multiplicity parameter. 
Define 

P,m (z h v , Tj) , 

an ideal of the integral domain R[z}. 
Theorem 4. Suppose f G I v ,m 

is nonzero. Let w = deg„(/). If c is a codeword of 

C u satisfying 

d(v, c) < n — w/m, 
then f(p) — 0, where p is the message function corresponding to c. 
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Proof. Let t = d(v, c). Assume /(/i) is not zero in R. Then 
w = deg u (/) > -v Poo (f(n)) 



= dim F (H//( M )) 

= dim w (R[z]/(f,z - fj,)) 

> dim ¥ (R[z]/{{z - h v ,r)) m + (z - m» 



m(n — t). 



Therefore if m(n — t) > w, we must have /(/x) = 0. 



□ 



The first step of list decoding of Hcrmitian codes is to construct a nonzero / in 
I vm . The second step is to find roots of / over R, and output the list of message 
functions corresponding to codewords of C u . To maximize the possibility that the 
list contains the original message function corresponding to the sent codeword, / 
should be chosen such that the u-weighted degree of / is minimized, according to 
Theorem 0] 

4. Using Grobner Bases of Modules 

We call the elements in the set 

Q = {x l y 1 z k | < i, Q < j < q - 1, < k} 

monomials of R[z\. Recall that every element of R[z] can be written as a unique 
linear combination over F of monomials of R[z]. Note that 

&eg u {x l y 3 z k ) =qi + (q + l)j + uk. 

For two monomials x H y^ 1 z , x 12 y^ 2 z k2 in O, we declare 



if &eg u (x 11 yi 1 z kl ) > deg u (x 12 y^ 2 z k2 ) or k-y > &2 when tied. It is easy to verify 
that > u is a total order on O. Notions such as the leading term and the leading 
coefficient of / e R[z] are defined in the usual way. For / S R[z], the z-degree of 
/, written z-deg(/), is the degree of / as a polynomial in z over R. 

Now we define the Q-polynomial of I v , m as the unique, up to a constant multiple, 
element in I Vt m with the smallest leading term with respect to > u . By the definition, 
the Q"P°ly nom i a l is an element of I v ,m with the smallest it-weighted degree, and 
moreover it has the smallest z-degree among such elements. Therefore we may 
say that the Q-polynomial is an optimal choice for the interpolation step of list 
decoding, and that the goal of the interpolation step is to find the Q-polynomial 
efficiently. We now present our strategy for this task in the following. 

Let Q denote the Q-polynomial of I v , m from now on. Let I be a positive integer 
such that z-deg(Q) < I. We call I the list size parameter. Define 



Note that R[z]i is a free module over R of rank I + 1 with a free basis 1,2,..., z l . 
Define I v ^ m ,i = Iv.m H i?[z]/. Clearly I v>m .i is a submodulc of R[z]i over R. 

Proposition 5. I v ,m,l; as a module over R, has a set of generators consisting of 
G%, < i < I, where 



R[z]i = {fe R[z] | z-deg(/) < I}. 




< i < m, 
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Proof. Recall that 7„ ;TO is generated by Gi, < i < m as an ideal of R[z]. Note 
that for < i < m, 

z(z - h v y V m - 1 = V (z - h v ) l+1 r ] m - i - 1 + h v (z - h v yi) m -\ 

Using this equation repeatedly, we may write any / g I v ,m,l as a linear combination 
of the Gi with coefficients in R. Then since z-deg(/) < I, the coefficient of Gi for 
i > I in the linear combination must be zero. This completes the proof. □ 

Observe that the ring R = ¥[x,y] is in turn a free module over ¥[x] of rank 
q, with a free basis {1, y, . . . , y^ 1 }. This can be seen easily from the relation 
y q = —y + x g+1 . So we may view R[z]i as a free module of rank q(l + 1) over ¥[x] 
with a free basis {y^z 1 \ < i < 1,0 < j < q — 1}. The elements of l~l R[z]i will be 
called monomials of It is clear that the total order > u is precisely a monomial 

order on the free module R[z]i over ¥[x\. We also view I Vtm ,i as a submodule of the 
free module R[z]i over F[x]. A set of generators for I Vj rn,ii 

as a module over W[x], is 

{y j Gi \0<i<l,0<j <q-l}. 

It is immediate that the Q-polynomial of I v ^ m is also the element of l v;m _i with 
the smallest leading term with respect to >„. As a consequence of the definition 
of Grobner bases, Q occurs as the smallest element in any Grobner basis of the 
module I v ^ m ,i over ¥[x] with respect to >„. Unlike the computation of Grobner 
bases of ideals, it turns out that the computation of a Grobner basis of the module 
Iv,md over F[x] can be done efficiently. 

Example (continued). Suppose the received vector is 

v = (a 2 , 0,0, a 2 , 0,0, 0,0). 

Our multiplicity parameter is m = 2. Then 

h v = a 2 x 2 y + ax 3 + o?xy + x 2 + o?y + x + a 2 , 

and r] = x* + x. It will turn out that z-dcg(Q) = 2. So we take I = 2 as our list 
size parameter. As a module over R = F 4 [a;, y] with y 2 = y + x 3 , 

Iv,2,2 = (z + h v ,r]} 2 = (r] 2 ,riz + r]h v ,z 2 + hl) = (G , G\, G 2 ), 

where 

Go = x 8 + x 2 , 

Gi = (x 4 + x)z + (a 2 x 6 + a 2 x 5 + a 2 x A + a 2 x 3 + a 2 x 2 + o?x)y 

+ ax 7 + x 6 + x 5 + x 4 + x 3 + x 2 + a 2 x, 
G 2 = z 2 + (ax 4 + ax 2 + a)y + ax 7 + a 2 x 6 + ax 5 + x 4 + ax 3 + x 2 + a. 

As a module over ¥ 4 [x], I v ,2,2 = (Go,yGo,Gi,yGi,G2,yG2), where 
yG = {x s +x 2 )y, 

yG\ = (x 4 + x)yz + (ax 7 + ax 6 + ax 5 + ax 4 + ax 3 + ax 2 )y 

+ a 2 x 9 + a 2 x s + a 2 x 7 + a 2 x 6 + a 2 x b + a 2 x 4 , 
yG 2 = yz 2 + (ax 7 + a 2 x 6 + ax 5 + a 2 x 4 + ax 3 + a 2 x 2 )y + ax 7 + ax 5 + ax 3 . 
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5. An Interpolation Algorithm 

We obtain an interpolation algorithm for Hermitian codes, applying Algorithm 
G in the appendix to the free module R[z]i over ¥[x] and the set of generators y 3 Gi 
of the submodule I v ^ m ^ of R[z]i as given in the previous section. 

Let T = | < i < 1,0 < j < q - 1}. Tuples in T are ordered lexico- 

graphically. So (0, 0) is the first tuple in T and the successor of (i, j) is (i, j + 1) if 
j < q — 1 or (i + 1, 0) if j = q — 1. Thus {y 3 z l \ e T} is a basis for R[z]i as an 
F[x]-module and the weight of the basis element y 3 z l is ui + (q + The index of 
/ € R[z]i is the largest tuple (i, j) such that the coefficient of y 3 z l is nonzero. So if 
the leading term, with respect to >„, of / <G R[z]i is x l y 3 z k , then ind(lt(/)) = (k, j). 
Notice that \nd(y 3 Gi) = 

Algorithm I. The algorithm finds the element of I V:in .i with the smallest leading 
term. Initially set gij <— y 3 Gi for (i, j) G T. Let 

9i,j = a i,i,i',j'y^ z 

(i'd')eT 

during the execution of the algorithm. For r = (r 1 ,r 2 ) and s = (s\, s 2 ) in T, the 
abbreviation o rjS denotes Or 1; r 2 ,si,s 2 • 

11. Set r <- (0,0). 

12. Set r to the successor of r. If r e T, then proceed; otherwise go to step 16. 

13. Set s <— ind(lt(<? r )). If s = r, then go to step 12. 

14. Set d <— dcg(a r . s ) — dcg(a s . s ) and c <— lc(a riS ) lc(a s . s ) _1 . 

15. If d > 0, then set 

9r <- ffr - cx d g s . 
If d < 0, then set, storing g s in a temporary variable, 

9s ^ 9r, g r ^ x~ d g r - cg s . 

Go back to step 13. 

16. Output gij with the smallest leading term, and the algorithm terminates. 

The idea of the algorithm is to update the set of generators until ind(lt(<7 r )) = r 
for all r e T, in which case the updated set of generators is a Grobner basis of 
Iv,md trivially by Buchberger's criterion. 

Example (continued). We demonstrate the algorithm by finding the Q-polynomial 
of I v _2- In the following, each column corresponds to an element (i, j) of T, ordered 
from right to left. Each entry is a multiple of y 3 z l , and the coefficient polynomial 
from ¥[x] is parenthesized with only the leading term shown. 
After initialization, we have 



ffo.o = 










(x 8 + ■ 




5o,i = 






(x 8 + ■ 


■)y 






fll.O = 




(x 4 + ■ ■ 


■)z + (a 2 x 6 + - 


■)y + 


(ax 7 + ■ 





= 


(x 4 + • 


■)yz + 


(ax 7 + ■ 


■)y + 


(aV + ■ 




32,0 = 


z 2 + 




(ax 4 + ■ 


■)y + 


(ax 7 + ■ 





32,i = yz 2 - 


h 




(ax 7 + ■ 


•)y + 


(ax 7 + ■ 





For r = (0,0) and (0,1), already ind(lt(g r )) = r. So r proceeds to (1,0). When 
r = (1, 0), we find s = (0, 1) in step 13. Since d = —2, we update 31,0 and g ,i in 
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the second way in step 15. Then we get 



(x 4 + ---)yz + 



30.0 = 

30.1 = 

31.0 = 

31.1 = 

32.0 = 2 2 + 

32.1 = yz 2 + 

Now we find s = (0, 0) in step 13. Since d 
in the first way in step 15. Then we get 











(X* 


+ 


(x 4 + ■■■)« 


+ (a 2 x 6 + ■ 


■)y 


+ 


(ax 7 


+ 


(x 6 + -..)z 


+ (oV + - 


■)v 


+ 


(ax 9 


+ 


(ax 7 + ■ 


■)v 


+ 


(a 2 x 9 


+ 




(ax 4 + ■ 


■)v 


+ 


(ax 7 


+ 




(ax 7 + ■ 


■)v 


+ 


(ax 7 


+ 


2, this time 


5i,o and 50,1 


are 


updated 





3o,o = 














(x s + 


3o,i = 




(x 4 + -- 


■)z + 


(a 2 x 6 + ■ 


■)y 


+ 


(ax 7 + 


3i,o = 




(x 6 + -- 


•)* + 


(a 2 x 7 + ■ 


■)v 


+ 


(x s + 


3i,i = 


(x 4 + - 


■)yz + 




(ax 7 + ■ 


■)y 


+ 


(a 2 x 9 + 


32,0 = 


z 2 + 






(ax 4 + ■ 


■)y 


+ 


(ax 7 + 


32,1 = yz 2 - 


\- 






(ax 7 + ■ 


■)y 


+ 


(ax 7 + 



Now we find s = (0, 1) in step 13. Since d = —1, we update gi t o and go,i once again 
in the second way in step 15. Then we get 



30.0 = 

30.1 = 

31.0 = 

31.1 = 

32.0 = 

32.1 = 



(x 4 + ---)yz + 



(x 4 + - 
(x 6 + - 



')z 
■)z 



+ 
+ 



(a 2 x 6 H )y + 



(ax 7 + ■ 
(ax 4 + ■ 



)y 
■)y 



+ 
+ 



(ax 7 H )y + 



(x* 
(ax 7 
(a 2 x 8 
(a 2 x 9 
(ax 7 
(ax 7 

So 



z< + 

yz 2 + 

Finally we find s = (1, 0) = r in step 13. That is, ind(lt(<7 r )) = r for r = (1, 0) 
r is set to the next element in T in step 12. The algorithm proceeds in this way 
until ind(lt(<7 r )) = r for all r G T. When we reach step 16, we have the following 
Grobner basis of I v ,2,2- 



3o,o = 






z 2 


+ 


3o,i = 




(x + - 
(a 2 x 2 + ■ 




+ 


3i,o = 


yz 2 + 


■)z 2 


+ 


3i,i = 










32,0 = 




(x 2 + - 


■> 2 


+ 


32,1 = (x + ■ 


■)yz 2 + 


(ax 2 + ■ 


■)z 2 


+ 



(a 2 x 5 H )z 

(x 4 + ■ ■ ■ )yz + (a 2 x 5 + ---)z 
(a 2 x 4 + •••)« 



(ax 4 + 
(ax 5 + 
+ (a 2 x 5 + 



)y 
)y 
■)y 



+ 
+ 



(ax 
.2^,6 



+ ■■•) 



(a 2 x 5 + ■■■)y + 



(a 2 x b + ■ 
(x 7 + ■ 



(Here the output for each <jfjj is broken into two lines.) Comparing the leading 
terms in step 16, we find that 32,0 is the smallest among the generators. Therefore 
the algorithm output 

Q = (x 2 + x)z 2 + {a 2 x A + a 2 x)z, 

which has factorization 

Q = (x 2 + x)z(z + a 2 x 2 + a 2 x + a 2 ). 
Hence a root-finding algorithm will output the list of roots 

0, a 2 x 2 + a 2 x + a 2 , 
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the second of which is the message function corresponding to the original codeword 
sent through the channel. 

Proposition 6. Aside from the computation of the initial set of generators, an 
execution of Algorithm I requires I 3 ) multiplication operations in F. 

Proof. We rely on Proposition El in the appendix. Note that 

v* G < = E (fc) (-i^W 

for < i < m, < j < q — 1. Since 

deg u (ri m - i hi- k y^z k ) < (m - i)q 3 + (i - k)(q 3 + q 2 - q - 1) + j(q + 1) + ku, 
we see 

deg„(^Gi) < m(q 3 + q 2 - q - 1) + q 2 - 1 

and 

deg u (?7 m ~y z l ) = (m - i)q 3 + j(q + 1) + iu > mu. 
Hence, according to Proposition an execution of the algorithm requires 

0((q 3 + q 2 -q- l)q^{q 3 + q 2 - q - 1 - u)q 3 m 2 l 3 ) = 0{q 8 m 2 l 3 ) 
multiplication operations in F. □ 

6. Upper bounds for the Q-polynomial 

We obtain simple upper bounds on the it-weighted degree and the z-degree of 
the Q-polynomial of I v _ rn . The u- weighted degree of Q determines the number of 
errors that the list decoder can correct. The z-degree of Q is used to set the list 
size parameter for the list decoder. 

Proposition 7. If I C Q is a finite set of monomials of R[z] such that 

then there is a set of coefficients c v € F such that 

7^ ^2 c v ip £ I v>m . 

tp£l 

Proof. The first assertion of Lemma [21 implies that monomials in I are linearly 
dependent over F in R[z]/I v m . On the other hand, they are linearly independent 
over F in R[z]. This completes the proof. □ 

In a table, we arrange monomials of R[z] such that the monomials in the same 
column have the same it-weighted degree and the monomials in the same row have 
the same z-degree. Let weighted degrees increase from left to right and z-degrecs 
from bottom to top. 

Example (continued). Recall that q = 2, u = 4. So deg u {x l z k ) = 2i + 3j + 4k. 



3 


























z 3 


o ••• 


2 


















z 2 





2 

XZ 


9 

yz 


2 2 

X z z 


xyz 


1 










z 


o 


xz 


yz 


2 

x z z 


xyz 


X 3 Z 


x 2 yz 


4 
X Z 


x 3 yz 





1 





X 


y 


x 2 


xy 


X 3 


x 2 y 


x 4 


x 3 y 


X 5 


4 

v V 


X 6 


x 5 y 







i 


2 


3 


4 


5 


6 


7 


8 


9 


10 


11 


12 


13 
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The symbol Q indicates that there is no monomial for the position. 

The table of monomials of R[z] suggests the following formula. Let G(i) = if 
i is a Weierstrass gap at P^, and 1 otherwise. Note that G(i) — 1 for i > 2g. The 
number of monomials with u- weighted degree i is 

Li/uj 

C(i) = £ G(* - u 3)- 

3=0 

Let w be the smallest integer such that 

^=n( m 2 +i ) +1 <f:^). 

Let I = [w/u\. Then the u- weighted degrees and the z-degrees of monomials up 
to the .ZVth monomial are not greater than w and I, respectively. Now Proposition 
implies deg u (Q) < w and z-deg(Q) < I. Theorem 0] guarantees the list decoder 
with these parameters to, I will correctly decode (that is, the list of roots contains 
the original message function) when there are at most \n — w/m~\ — 1 errors. 



Example (continued). G(0) = 1,G(1) = 0, and G(i) = 1 for i > 2 since g = 1. 
Recalling that u = 4, we have 



i 





1 


2 


3 


4 


5 


6 


7 


8 


9 


10 


11 


12 


13 


14 


15 




C(i) 


1 





1 


1 


2 


1 


2 


2 


3 


2 


3 


3 


4 


3 


4 


4 




e;= c(j) 


1 


1 


2 


3 


5 


6 


8 


10 


13 


15 


18 


21 


25 


28 


32 


36 





For to = 2, N — 25. So w = 12, and I = 3. By the argument above, the list decoder 
with parameters to = 2, / = 3 is guaranteed to decode one arbitrary error. 

Using the same bounds, successful decoding for two arbitrary errors is guaranteed 
if we take parameters to = 6,1 — 8. Thus the successful decoding of two errors in 
the example with parameters to = 2, 1 = 2 is not to be expected from the bounds 
we have. In fact, our experiments show that decoding failures for two errors with 
parameters to = 2, 1 = 2 are actually infrequent. We expect that the bounds given 
above significantly underestimate the capability of the algorithm. 

7. Concluding Remarks 

We formulated list decoding of Hermitian codes anew, and presented a simple 
and efficient algorithm for the interpolation step. It is not easy to compare fairly our 
interpolation algorithm with previously known algorithms |14l l7l H3lll()j . However, 
our algorithm has a good computational complexity while its simple description 
affords a straightford hardware implementation. 

The interpolation algorithm for Reed-Solomon codes in 9\ was shown to be equiv- 
alent to the Berlekamp-Massey algorithm in the special case when the multiplicity 
and list size parameters are all one. We expect that there is also an intimate rela- 
tion between our interpolation algorithm for Hermitian codes with multiplicity and 
list size parameters all one and Kotter's version of the Berlekamp-Massey-Sakata 
algorithm PITT]. 

Present bounds for the Q-polynomial need to be improved. In experiments, our 
list decoder with certain multiplicity and list size parameters shows a better rate 
of successful decoding than would be expected from the present bounds. A better 
understanding of the capability of the list decoder is required. 
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Though we try to make our formulation of list decoding as independent as pos- 
sible from special properties of Hcrmitian codes, it is not clear what is the most 
general class of algebraic geometry codes for which list decoding is possible in a 
similar fashion. 

Appendix A. A Grobner Basis Algorithm 

We consider a submodule S oik[x] m . Let e\ < e 2 < ■ ■ ■ < e m denote the standard 
basis of fc[x] m . Let u = (u x ,ui,u 2 , ■ ■ ■ , u m ) be a given sequence of positive integers. 
The u- weighted degree of a monomial x r ei is defined to be deg u (a; r e i ) = u x r + Ui. 
Thus deg u (aei) = u x deg(a) + ui for a G k[x\. A monomial order >„ of fc[x] m is 
defined by declaring x r ei > u x s ej if deg u (a; r e i ) > deg u (x s ej) or if i > j when the 
weighted degrees are tied. 

For / = Y^iiLi a i e i w hh ai <G k[x], define the index of /, written ind(/), to be 
the largest i such that ai ^ 0. In particular, ind(x r ei) = i. 

Suppose {G\, G2, ■ ■ ■ , G m } is a set of generators of the module S such that 
ind(Gi) = i. Then the following algorithm computes a Grobner basis of S from the 
given set of generators with respect to the monomial order >„. 

Algorithm G. Let gi = Y^jLi a ij e j for 1 < z < m during the execution of the 
algorithm. Initialize with gi <— Gi. 

Gl. Set r <- 1. 

G2. Increase r by 1. If r < m, then proceed; otherwise go to step G6. 
G3. Set s <— ind(lt(g r )). If s = r, then go to step G2. 
G4. Set d <— deg(a rs ) — deg(a ss ) and c <— lc(a rs ) lc(a ss ) _1 . 
G5. If d > 0, then set 

g r <-g r - cx d g s . 
If d < 0, then set, storing g s in a temporary variable, 

9s ^ 9r, 9r <- x~ d g r - cg s . 

Go back to step G3. 
G6. Output {<?i, . . . , g m } and the algorithm terminates. 

Proposition 8. For each 1 < r < m, it occurs that ind(lt(gi)) = i for 1 < i < r 
after a finite number of iterations through the steps G3-G5. 

Proof. We actually prove that the following hold after initialization and after the 
iteration steps G3-G5: 

(i) ind(<?i) < r for 1 < i < r. 

(ii) ind(lt(5i)) = i for 1 < i < r - 1. 

(hi) for every non- identity permutation n of {1, 2, . . . , r}, 

r r 

^deg 

1=1 1=1 

After initialization in step Gl, when r = 1, items (i)-(iii) are true. After r is 
increased by one in step G2, (i) and (ii) clearly hold; (iii) also holds for the case 
7r r = r because it holds for the previous value of r, and for the case ir r ^ r because 
deg u (ai 7ri e 7r J = —00 for the i such that 7Tj = r. It remains to check (i)-(iii) after 
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the update in step G5. Item (i) is clear. Item (ii) still holds because leading terms 
of both g s and g r have index s. Explicitly 

(!) deg u (a si ej) < deg M (a ss e s ) > deg u (a sj >ef), 

(2) deg u (a rj ej) < deg u (a rs e s ) > deg u (a rj ,e r ) 

for r > j > s > f > 0. Using (JTJ, ©, and (hi), we can prove Propositions and 
ITTI Then using Al, Bl, (ii), and (hi), we can prove Propositions 1101 and 1121 which 
show that (hi) still holds after the update in step G5. 

Finally Propositions [§] and ^2 show that after the update in step G5, either 
deg M (lt(<jv)) — deg u (a rr e r ) strictly decreases or else the index of lt(g r ) strictly de- 
creases. Therefore it will eventually happen that ind(lt(g,.)) = r, which together 
with (ii) completes the proof. □ 

Proposition 9. Assume the case d > 0. Let 

where b r j = a r j — cx d a s j for < j < r. Then the following hold. 

Al. deg„(6„.e r ) = deg u (a rr e r ). 

A2. deg u (6 rs e s ) < deg u ( 

A3. deg u (6 r . J e i ) < deg u (a rs e s ) for s <j < r. 

A4. deg u (b rj ej) < deg u (a rs e s ) for j < s. 

In particular, 

degjlt(g)) - deg„(6 rr e r ) < deg u (lt(g r )) - deg u (a rr e r ), 
where the equality holds only i/ ind(lt(g)) < ind(lt(<7 r )). 

Proof. From (iii), choosing for it the transposition of s and r, we have deg(a rr ) + 
deg(a ss ) > deg(a rs ) + deg(a sr ). Then Al follows since 

deg(x d a sr ) = deg(a rs ) - deg(a ss ) + deg(a sr ) < deg(a rr ). 

A2 holds since c and d were chosen such that 

deg(b rs ) = deg(a rs - cx d a ss ) < deg(a rs ). 

Let s < j < r. By JTJ, 

deg u (x d a S3 ej) = deg u (a rs e s ) - deg u (a ss e s ) + deg u (a SJ e J ) < deg u (a rs e s ), 

which together with @ shows A3. Let j < s. Similarly by 

deg u (x d a SJ e j ) = deg u (a rs e s ) - deg u (a ss e s ) + deg u (a sj ej) < deg u (a rs e s ), 

which together with (J2J) shows A4. 

Now we show that the last assertion follows from A1-A4. Let j = ind(lt(<?)) so 
that lt(<?) = lt(b r jej). Recall that lt(g r ) = lt(a rs e s ). If j = r, then the assertion is 
obvious. Suppose j <r. Then by A2-A4, 

degjlt( 5 )) < deg n (lt(. 9r )), 

where the equality holds only if ind(lt(g)) < ind(lt(g r )). On the other hand, 
deg u (6 rr e r ) = deg u (a rr e r ) by Al. The assertion follows. □ 
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Proposition 10. Assume d > 0, and retain previous notations. For every non- 
identity permutation n = (tti, 7T2, ■ ■ ■ , 7r r ) of {1, 2, . . . , r}, 

( 3 ) ^2^Su( a u e i) + deg tl (6 rr e r ) > deg„(a i7ri e 7ri ) + deg„(6 r7rr e 7rr ). 
Proof. By Al, the left hand side of © equals 

r 

i=i 

Recall that b rnr = a r7Tr — cx d a S7Tr . Hence deg(6 r7rr ) < deg(a r7Tr ) or deg(6 r7rr ) = 
deg(a rs ) - deg(a ss ) + deg(a sjrr ). 

If deg(b r7Tr ) < deg(a r7Ir ), then the right hand side of © is 

< y^deg tt (afa j e 7r J + deg tl (a r7rr e Trr ) = y^deg„(aj 7r( e 7r J < y^deg M (a ri eQ, 

i^r i i 

where the last inequality holds by (iii). Thus © holds. 

Now we consider the case when deg(b r7rr ) = deg(a rs ) — deg(a ss ) +deg(a S7Tr ). Let 
Dij denote deg u (ciy e.,). Then © is equivalent to 

(4) Yj Du> H D ™* + + D ™r + Drs ~ Dss - 

To show we treat two cases depending on whether s and 7r r are in the same 
orbit or not, with respect to the permutation ir. First suppose s and n r are in the 
same orbit so that 

7T r ► 7T(7TY j ► • • • ► 7T (Sj ► S. 

Let 5* = {ir r , 7r(7r r ), . . . ,7r _1 (s)}. Note that S is empty if 7iy = s. Now the right 
hand side of 10} equals 

D l7!t + D ™i + D ^s + D rs - D ss + D S7Tr 

(5) 

< ^ Du + D ™* + D ^s + D rs- 

ieS i$_S,i=£s,r 

This inequality holds since Da > for 1 < i < r — 1 and 1 < j ' < r by (ii). We 
can check that the second indices of the terms in the last expression of © are all 
distinct. So by (iii), we have 

Y D u + Y D ™* + +D rs <Y D ll - 

i£S igS,i^s,r i 

Hence Q is proved. 

The diagram (a) in Figure ^ below gives an example exhibiting the intuition 
behind the argument above, for the case when s and ir r are in the same orbit. In 
the diagram, the smaller circles mark the terms in the first sum of (JjjJ and the 
larger circles mark those of the second sum. Similar diagrams will be helpful later. 

If s and 7r r are not in the same orbit, then we have 

7T S ► 7r(7r,j) > • • • ► 7T (Sj > S, 
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Figure 1. 



and let 5 = {tt s , tt(tt s ), . . . , ir 1 (s)}. Note that S is empty if ir s = s. Now the right 
hand side of equals 

where the inequalities are justified by similar arguments as above. See the diagram 
(b) in Figure □ □ 

Proposition 11. Assume the case d < 0. Let 

g = x~ d g r — cg s — b rr e r + • ■ ■ + b rs e s + ■ • • , 
where b r j — x~ d a r j — ca S j for < j < r. Then the following hold 

Bl. deg u (b rr e r ) = deg u (x~ d a rr e r ). 

B2. dcg u (b rs e s ) < deg u (x~ d 

B3. dcg u {b r] ej) < deg u (x~ d a rs e s ) for s < j <r. 

B4. deg u (b rj ej) < deg u (x~ d a rs e s ) for j < s. 
In particular, 

d eg„(h(3)) - deg u (6 rr e r ) < deg u (h% r )) - deg u (a rr e r ), 
where the equality holds only if ind(lt(gr)) < ind(\t(g r )) . 
Proof. Bl holds since 

deg(x~ d a rr ) = deg(a ss ) - deg(a rs ) + deg(a rr ) > deg(a sr ) 
by (iii). B2 holds because c and d were chosen such that 

deg(6 rs ) = deg(x~ d a rs - ca ss ) < deg(x~ d a rs ). 
Let s < j < r. By JTJ and @, 

^g u (x~ d a rj ej) < deg u (x~ d a rs e s ), 
deg u (a sj ej) < deg u (a ss e s ) = deg u (x~~ d a rs e s ), 
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from which B3 follows. Let j < s. Again by (JJJ and J3J, 

de E u ( x ~ da rj e j) < deg u (x~ d a rs e s ), 

de g«( a si e i) < deg u (a ss e s ) = deg u (x~ d a rs e s ), 

from which B4 follows. 

Now we show that the last assertion follows from B1-B4. Let j = ind(lt(p)) so 
that lt(g) = \t(b r jej). Recall that \t(g r ) — lt(a rs e s ). If j = r, then the assertion is 
obvious. Suppose j < r. Then by B2-B4, 

degjlt(g)) <deg u (x~ d lt(g r )), 

where the equality holds only if ind(lt(g)) < ind(lt(<7 r )). On the other hand, 
deg u (6 rr e r ) = deg u (x~ d a rr e r ) by Bl. The assertion follows. □ 

Proposition 12. Assume the case d < 0, and retain previous notations. For every 
non-identity permutation tt = (tti, 7T2, . . . , 7T r ) of {1, 2, . . . , r} ; 

( 6 ) de g«( a « e + deg u (a rs e s ) + deg u (6 rr e r ) 

> 2^ deg u (ai^-e^) + deg u (a r7rs e 7r J + deg u (6 r7rr e 7rr ). 

Proof. By Bl, deg u (& rr e r ) = ii^(deg(a ss ) — deg(a rs )) + deg u (a rr e r ). Therefore the 
left hand side of © equals 

r 

^2deg u (a u ei). 

i=i 

Note that b Hence deg(6 rirr ) < deg(a SJIV ) or deg(6 r7r J = 

deg(a ss ) - deg(a rs ) + deg(a r7r ,.). 

If deg(6 rOTr ) < deg(a S7I> ), then the right hand side of iJBJ is 

< ^ deg„(a i7ri e 7ri ) + deg„(a rjra e w J + deg u (a SWr e Wr ) < ^ deg u (a li e i ), 

where the last inequality holds by (iii). Thus © holds. 

Now we suppose deg(b rTr ) = deg(et ss ) — deg(a rs ) + deg(a rTr ). Let Dij denote 
deg„(a i je J ). Note that © is equivalent to 

(7) D " > Yl D ™> + D 

r7r s ~i~ D r7Tr -\- D ss D rs . 

To show this, we treat two cases depending on whether s and ir r are in the same 
orbit or not, with respect to the permutation tt. First suppose s and ir r are in the 
same orbit so that 

7T r > 7T(7T r j > • • ■ > TT [S) > S. 

Let S — {ir ri 7r(7r r ), . . . ,7r -1 (s)}. Note that 5 is empty if u r = s. Now the right 
hand side of Q equals 

< 2^ Da + 2^ Att, + D rWs + D s 

ieS igS,ijis,r 
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This inequality holds since Du > Dij for 1 < i < r — 1 and 1 < j < r by (ii) and 
that D rs > D r j for 1 < j < r by the way in which s is chosen. We can check that 
the right indices of terms in the final expression of JHJ are all distinct. So by (iii), 
we see that 

J2 d u+ D *** + D ™* +d ss <y^ d u . 

Hence (J7J) is proved. See the diagram (c) in Figure |21 
If s and ir r are not in the same orbit, then we have 

7T S > 7T(7T s ) > ■ ■ ■ > 7T (S) ► S, 

and let S = {ir s , 7r(7r s ), . . . , 7r _1 (s)}. Note that S is empty if ir s = s. Now the right 
hand side of (J7J equals 

i£S i$_S,i=£s,r 

< ^ D u + d **< + B ^r +D SS <Y D u , 

where the inequalities hold by the same reasons as above. See the diagram (d) in 
Figure |2 
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Figure 2. 

□ 

Proposition 13. Lei <?.; = J^,- = x a ij e ji 1 < i < rn be an input for the algorithm. 
Let c be an upper bound on deg u (aijej), 1 < i,j < m. Let d be an upper bound on 
deg u (aijej) — deg u (aaei), 1 < j < % < m. T/ien an execution of the algorithm for 
the input gi requres 0(cu~ 1 dm 3 ) multiplication operations in the field k. 

Proof. The proof of Proposition [3] implies that for each 1 < i < m, at most 
?(deg u (lt(<7i))— deg u (auei j) updates for <fa are performed. Note that d > deg„ (It ) ) — 
deg u (auei). Note also that each update requires at most cm" 1 ? multiplication op- 
erations in k. Therefore at most J2iLi cu x l ^ multiplication operations are re- 
quired. □ 
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